Welcome to the journey:
OHDSI Symposium 2015
Wifi:
Network: HHONORS-MEETING
Passcode: OHDSI15
Welcome to the journey:
Overview of OHDSI : past,
present, future
Patrick Ryan, PhD
Janssen Research and Development
Columbia University Medical Center
20 October 2015
Odyssey (noun): \oh-d-si\
1. A long journey full of adventures
2. A series of experiences that give knowledge or
understanding to someone
http://www.merriam-webster.com/dictionary/odyssey
A journey to OHDSI
Greetings to you, the lucky finder of this GOLDEN TICKET
from the OMOP Research Team
OMOP’S
Present this ticket at the next OMOP Symposium in the morning and do not
be late. You may bring with you one member of your own family…and only
one….but no one else…
In your wildest dreams, you could not imagine the marvelous
SURPRISES that await YOU!
G O L D E N T I C K E T
Thanks to our sponsors
Thanks for the Eugene Washington Engagement
Award
Lesson 1: Database heterogeneity:
Holding analysis constant, different data may yield
different estimates
Madigan D, Ryan PB, Schuemie MJ et al, American Journal of Epidemiology, 2013
“Evaluating the Impact of Database Heterogeneity on Observational Study Results”
Relative risk
Test cases from OMOP 2011/2012 experiment
Lesson 2: Parameter sensitivity:
Holding data constant, different analytic design
choices may yield different estimates
Madigan D, Ryan PB, Scheumie MJ, Therapeutic Advances in Drug Safety, 2013: “Does design matter?
Systematic evaluation of the impact of analytical choices on effect estimates in observational studies”
Lesson 3: Empirical performance:
Most observational methods do not have nominal
statistical operating characteristics
Ryan PB, Stang PE, Overhage JM et al, Drug Safety, 2013:
“A Comparison of the Empirical Performance of Methods for a Risk Identification System”
Lesson 4: Empirical calibration can help restore
interpretation of study findings
Schuemie MJ, Ryan PB, DuMouchel W, et al, Statistics in Medicine, 2013:
“Interpreting observational studies: why empirical calibration is needed to correct p-values”
Lesson 5: Reliable evidence generation isn’t (just) a
data/analysis/technology problem
• Understanding the problems requires input and perspective
from multiple stakeholders: government, industry, academia,
health systems
• Research and development of novel solutions require multi-
disciplinary approach: informatics, epidemiology, statistics,
clinical sciences
• Adoption and application requires active participation and
buy-in from all interested parties (both evidence producers
and evidence consumers)
• Major outstanding need: to establish a community of
individuals based on shared attitudes, interests and goals
where everyone has equal opportunity to participate and
contribute
Introducing OHDSI
• The Observational Health Data Sciences and
Informatics (OHDSI) program is a multi-
stakeholder, interdisciplinary collaborative to
create open-source solutions that bring out
the value of observational health data through
large-scale analytics
• OHDSI has established an international
network of researchers and observational
health databases with a central coordinating
center housed at Columbia University
http://ohdsi.org
OHDSI: a global community
OHDSI Collaborators:
• >100 researchers in academia, industry,
government, health systems
• >10 countries
• Multi-disciplinary expertise: epidemiology,
statistics, medical informatics, computer
science, machine learning, clinical sciences
http://www.ohdsi.org/who-we-are/collaborators/
Global reach of ohdsi.org
• >16,800 distinct viewers from 120 countries in 2015
Page 18
The journey of the OMOP Common data model
OMOP CDMv2
OMOP CDMv4
OMOP CDMv5
OMOP CDM now Version 5, following
multiple iterations of implementation,
testing, modifications, and expansion
based on the experiences of the
community who bring on a growing
landscape of research use cases.
Concept
Concept_relationship
Concept_ancestor
Vocabulary
Source_to_concept_map
Relationship
Concept_synonym
Drug_strength
Cohort_definition
Standardized vocabularies
Attribute_definition
Domain
Concept_class
Cohort
Dose_era
Condition_era
Drug_era
Cohort_attribute
Standardized
derived elements
Standardized clinical data
Drug_exposure
Condition_occurrence
Procedure_occurrence
Visit_occurrence
Measurement
Procedure_cost
Drug_cost
Observation_period
Payer_plan_period
Provider
Care_site Location
Death
Visit_cost
Device_exposure
Device_cost
Observation
Note
Standardized health system data
Fact_relationship
Specimen
CDM_source
Standardized meta-data
Standardized health
economics
Drug safety surveillance
Device safety surveillance
Vaccine safety surveillance
Comparative effectiveness
Health economics
Quality of care
Clinical research
One model, multiple use cases
Person
The odyssey to evidence generation
Patient-level
data in source
system/ schema
evidence
Preparing your data for analysis
Patient-level
data in source
system/ schema
Patient-level
data in
OMOP CDM
ETL
design
ETL
implement
ETL test
WhiteRabbit:
profile your
source data
RabbitInAHat:
map your source
structure to
CDM tables and
fields
ATHENA:
standardized
vocabularies
for all CDM
domains
ACHILLES:
profile your
CDM data;
review data
quality
assessment;
explore
population-
level summaries
OHDSI tools built to help
CDM:
DDL, index,
constraints for
Oracle, SQL
Server,
PostgresQL;
Vocabulary tables
with loading
scripts
http://github.com/OHDSI
OHDSI Forums:
Public discussions for OMOP CDM Implementers/developers
Usagi:
map your
source codes
to CDM
vocabulary
Single study
Real-time query
Large-scale analytics
Data Evidence sharing paradigms
Patient-level
data in
OMOP CDM
evidence
Write
Protocol
Develop
code
Execute
analysis
Compile
result
Develop
app
Design
query
Submit
job
Review
result
Develop
app
Execute
script
Explore
results
One-time Repeated
What evidence does OHDSI seek to
generate from observational data?
• Clinical characterization
– Natural history: Who are the patients who have diabetes? Among
those patients, who takes metformin?
– Quality improvement: what proportion of patients with diabetes
experience disease-related complications?
• Population-level estimation
– Safety surveillance: Does metformin cause lactic acidosis?
– Comparative effectiveness: Does metformin cause lactic acidosis
more than glyburide?
• Patient-level prediction
– Precision medicine: Given everything you know about me and my
medical history, if I start taking metformin, what is the chance that I
am going to have lactic acidosis in the next year?
– Disease interception: Given everything you know about me, what is
the chance I will develop diabetes?
What is OHDSI’s strategy to generate
evidence?
• Methodological research
– Develop new approaches to observational data analysis
– Evaluate the performance of new and existing methods
– Establish empirically-based scientific best practices
• Open-source analytics development
– Design tools for data transformation and standardization
– Implement statistical methods for large-scale analytics
– Build interactive visualization for evidence exploration
• Clinical applications
– Identify clinically-relevant questions that require real-world evidence
– Execute research studies by applying scientific best practices through
open-source tools across the OHDSI international data network
– Promote open-science strategies for transparent study design and
evidence dissemination
Methodological research
Open-source
analytics
development
Clinical applications
Observational
data management
Population-level
estimation
Patient-level
prediction
• Data quality assessment
• Common Data Model evaluation
• ATHENA for standardized
vocabularies
• WhiteRabbit for CDM ETL
• Usagi for vocabulary mapping
• HERMES for vocabulary exploration
• ACHILLES for database profiling
• CohortMethod
• SelfControlledCaseSeries
• SelfControlledCohort
• TemporalPatternDiscovery
• PatientLevelPrediction
• APHRODITE for predictive
phenotyping
• Empirical calibration
• LAERTES for evidence synthesis
• PENELOPE for patient-centered
product labeling
• Chronic disease therapy pathways
• HOMER for causality assessment
Clinical
characterization
• CIRCE for cohort definition
• CALYPSO for feasibility assessment
• HERACLES for cohort
characterization
• Phenotype evaluation
• Evaluation framework and
benchmarking
OHDSI ongoing collaborative activities
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Observational
data management
Methodological research
Poster:
Establishing Interoperability Standards
between OMOP CDM v4, v5, and
PCORnet CDM v1
PCORNet CDRNs:
• NYC CDRN
• PEDSNet
• pSCANNER
Common standard
reference model:
OMOP CDM
Common ETL for
project destination:
PCORNet CDM
OHDSI ongoing collaborative activities
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Observational
data management
Methodological research
Poster:
Transforming the National Department
of Veterans Affairs Data Warehouse to
the OMOP Common Data Model
OHDSI ongoing collaborative activities
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Observational
data management
Methodological research
OHDSI Community Booth:
ETL 101
OHDSI ongoing collaborative activities
Open-source
analytics
development
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Observational
data management
OHDSI Community Booth:
ATHENA for standardized vocabularies
OHDSI ongoing collaborative activities
Open-source
analytics
development
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Observational
data management
Open-source analytic demo:
HERMES for vocabulary exploration
OHDSI ongoing collaborative activities
Open-source
analytics
development
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Observational
data management
Open-source analytic demo:
ACHILLES for database profiling
OHDSI ongoing collaborative activities
Open-source
analytics
development
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Observational
data management
Poster:
Determination of Pregnancy Episodes and
Outcomes within a Distributed Network of
Observational Databases
OHDSI ongoing collaborative activities
Clinical applications
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Poster:
Size comparison of 17 CDM
datasets using IRIS tool
OHDSI ongoing collaborative activities
Clinical
characterization
Methodological research
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Poster:
Lessons from CIRCE
implementation of eMERGE
phenotype definitions into
actionable CDM v5 SQL queries
OHDSI ongoing collaborative activities
Clinical
characterization
Methodological research
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Open-source analytics demos:
CIRCE for cohort definition
CALYPSO for feasibility
assessment
OHDSI ongoing collaborative activities
Clinical
characterization
Open-source
analytics
development
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Open-source analytics demos:
HERACLES for cohort
characterization
OHDSI ongoing collaborative activities
Clinical
characterization
Open-source
analytics
development
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Poster:
Exploration of the Epidemiology of Endometriosis
OHDSI ongoing collaborative activities
Clinical
characterization
Clinical applications
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Presentation:
Treatment pathways in chronic disease
OHDSI ongoing collaborative activities
Clinical
characterization
Clinical applications
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Open-source analytic demo:
LAERTES for evidence integration
OHDSI ongoing collaborative activities
Methodological research
Population-level
estimation
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Poster:
Accuracy of an Automated Knowledgebase for
Identifying Adverse Drug Reactions
OHDSI ongoing collaborative activities
Methodological research
Population-level
estimation
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Poster:
How high can we go? Evaluating
massively high-dimensional propensity
score and outcome models in large-scale
observational studies
OHDSI ongoing collaborative activities
Methodological research
Population-level
estimation
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Open-source analytics demos:
Cohort Method
Self-controlled case series
Empirical calibration
OHDSI ongoing collaborative activities
Population-level
estimation
Open-source
analytics
development
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Poster:
A Climate-Wide Journey to Explore
Mechanisms Underlying Birth
Month Disease Risk Associations
OHDSI ongoing collaborative activities
Population-level
estimation
Clinical applications
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Poster:
Discovering the hidden risk
factors: An empirical evaluation
of incorporating feature-
learning methods into a risk
model framework using the
OMOP CDM
OHDSI ongoing collaborative activities
Methodological research
Patient-level
prediction
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Open-source analytic demo:
APHRODITE for predictive
phenotyping
OHDSI ongoing collaborative activities
Patient-level
prediction
Open-source
analytics
development
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
Poster:
Lift your Anchors and Begin the
OHDSI with APHRODITE
OHDSI ongoing collaborative activities
Patient-level
prediction
Clinical applications
Methodological
research
Open-
source
analytics
developme
nt
Clinical
application
s
Observation
al data
managemen
t
Population-
level
estimation
Patient-level
prediction
Clinical
characteriza
tion
OHDSI ongoing collaborative activities
Methodological research
Open-source
analytics
development
Clinical applications
Observational
data management
Population-level
estimation
Patient-level
prediction
Clinical
characterization
Poster:
OHDSI Cloud Architecture
OHDSI commercial ecosystem
Journey through the OHDSI
collaborator showcase
Data
modeling
and
standard-
ization
E
T
L
1
0
1
APHRODITE
Clinical
applications
Methods
research
Analytics
development
ACHILLES
HERMES
ATHENA
HERACLES
LAERTES
CIRCE
OHDSI
101
E
c
o
s
y
s
t
e
m
Panel Discussion – Experiences from the
OHDSI international data network
Panel Discussion – The Value and Challenges
of Evidence from Observational Data: A
Multi-Stakeholder Perspective
• Moderator: David Madigan, PhD, Executive Vice
President and Dean of the Faculty of Arts and
Sciences at Columbia University
• Robert Ball, MD, MPH, ScM, Deputy Director – Office
of Surveillance and Epidemiology, CDER, US Food
and Drug Administration
• Invited: Robert Califf, MD, Deputy Commissioner of
Medical Products and Tobacco, US Food and Drug
Administration
• Nareesa Mohammed-Rajput, MD, Medical Director
of Clinical Informatics, Suburban Hospital part of
Johns Hopkins Medicine
• Maryan Zirkle MD, MS, MA, Program Officer – CER
Methods and Infrastructure Program, PCORI
• Lesley Wise, Vice President of PV Risk Management
and Pharmacoepidemiology, Takeda Pharmaceuticals
Future of OHDSI
This is your journey….
….where do we go from here?
Standardized large-scale analytics tools
under development within OHDSI
Patient-level
data in
OMOP CDM
http://github.com/OHDSI
ACHILLES:
Database
profiling
CIRCE:
Cohort
definition
HERACLES:
Cohort
characterization
OHDSI Methods Library:
CYCLOPS
CohortMethod
SelfControlledCaseSeries
SelfControlledCohort
TemporalPatternDiscovery
Empirical Calibration
HERMES:
Vocabulary
exploration
LAERTES:
Drug-AE
evidence base
HOMER:
Population-level
causality
assessment
PLATO:
Patient-level
predictive
modeling
CALYPSO:
Feasibility
assessment
PENELOPE:
Patient-
centered
product labeling
Thank you OHDSI Symposium
Organizing Committee
David Sontag
NYU
Ana Szarfman
FDA
Charlie Bailey
CHOP
Jon Duke
Regenstrief
Chunhua Weng
Columbia University
Gregory Fusco
Takeda
Thank you Maura Beaton
Join the journey
Interested in OHDSI?
Questions or comments?
Contact:
ryan@ohdsi.org
58
